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Abstract 

This paper evaluates the implementation of Learning Design on the production of a core FHEQ level 6 
(QAA, 2008) 1 unit of study at a UK distance learning institution. By comparing student (41=656) and 
tutor (n=42) survey data with questionnaire responses (n=9) from the unit of study’s core production 
team, this paper assesses the impact of incorporating the Open University Learning Design Initiative 
(OULDI) methodology into curriculum production by looking specifically at barriers and facilitators in 
the application of Learning Design and its impact on module development, delivery, and the resulting 
student and tutor learning experience. With a focus on developing and embedding Learning Design 
into the curriculum planning and production process, the paper explores how lessons learned from this 
experience have helped to guide and inform the future implementation of Learning Design into module 
and qualification level frameworks. 
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Literature Review 

Learning Design (LD) is the practice of planning, sequencing, and managing learning activities, usually 
using ICT-based tools to support both design and delivery. The aim of Learning Design is to describe 
pedagogic intentions in a particular learning context; providing “a broad picture of a series of planned 
pedagogical actions rather than detailed accounts of a particular instructional event” (Lockyer, 
Heathcote, & Dawson, 2013, p. 1442). 

Literature offers a number of different perspectives on what is meant by the term “Learning Design.” 
From one perspective, it refers to a research methodology, which “seeks to develop a descriptive 
framework for teaching and learning activities and to explore how this framework can assist educators 
to share and adopt great teaching ideas” (Dalziel et al., 2013, p.5). A second perspective is that of an 
educational practice methodology - a systematic approach to the development of a unit of learning that 


1 Frameworks for Higher Education Qualifications in England, Wales and Northern Ireland, produced by the Quality Assurance 
Agency for Higher Education (2008) - this framework provides a reference point for all Higher Education providers for the 
setting and assessing of academic standards. 
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supports the more effective integration of technologies for learning. Here the focus on “design” is seen 
as significant because it re-frames the scholarship of teaching in response to a context of new and 
rapidly changing pedagogies (Beetham & Sharpe, 2007). The third perspective sees LD as a change 
process mechanism. That is, an enabler for restructuring and informing institutional curriculum 
planning practices (Anderson, Bullen, Alltree, & Thornton, 2008; Salmon, Jones, & Armellini, 2008). 

In contrast to the field of Instructional Design, which draws primarily on cognitivist learning theories 
(Dick & Carey, 1996; Gagne & Driscoll, 1988) the field of LD is firmly positioned within a tradition of 
socio-cultural educational research. For example, LD references concepts such as Activity Theory 
(Vygotsky, 1978; Engestrom, 2015) in the notion of “mediating artefacts” for learning and design 
(Conole, Dyke, Oliver, & Seale, 2004). LD research also emphasises the importance of the social and 
contextual aspects of a student's learning experience (Laurillard, 2002; Oliver, Harper, Hedberg, Wills, 
& Agostinho, 2002; Conole, 2013). This emphasis is in line with a movement over the last 20 years in 
UK and Australian educational discourse towards more social, contextual and distributed models of 
learning - away from an “acquisition” metaphor towards a “participation” metaphor - where knowledge 
or “knowing about” is considered to be an inherently situated activity (Sfard, 1998). 


Teaching and Learning Context 

The OU is the largest provider of distance-learning, part-time undergraduate and postgraduate study 
in the United Kingdom, for which students may register without formal entiy qualifications. The OU’s 
pedagogical approach is somewhat different to face to face institutions. We describe this teaching 
method as “blended learning” (Bonk & Graham, 2006) in that it can take place in a face-to-face 
environment (through regular tutorials) but also at a distance (through on-line tutorials, forums, and 
one-to-one telephone calls). Associate Lecturers (ALs) are employed by the OU to deliver and teach at 
a distance the materials written by central academic teams (module teams) for students. There are 
increased expectations for central academics to play extended roles within the modules (or “chunks” of 
study) they have created - for example, delivering podcasts, moderating student conferences, and 
monitoring AL marking and feedback. Although we have a physical headquarters in Milton Keynes, the 
reality is that our students and ALs can be located anywhere in the world. 

LD is particularly pertinent to the distance learning context for two reasons. First, it is important that 
a broad framework for the structure and pedagogy of curriculum can be clearly transmitted to ALs and 
students, learning, and working at a distance. Second, it is more difficult for a distance learning provider 
to identify and fix problems in curriculum materials when a course is underway as print materials will 
already be in use by students and ALs. LD provides the module team with a way of identifying potential 
issues prior to materials being released, by using module maps and activity planners to visualise the 
likely teaching and learning experience (Galley, 2012). 


Method 


This paper evaluates a project trialling a new module design process which “frontloaded” the LD activity 
in advance of the start of module production on a popular and well regarded 60 credit module EK310 
Research with Children and Young People into a new module EK313 Issues in Research with Children 
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and Young People. This is a compulsory level 6 module (equivalent to the last year of undergraduate 
study) across four different qualification pathways and draws on students from a range of different 
discipline backgrounds, including Childhood and Youth Studies, Early Years, Youth Justice, and 
Primary Studies. In particular, this pilot sought to develop an approach which provided greater 
emphasis on pedagogy and support for LD, and that would lead to the design and development of a 
module which met the aims of the Faculty of Education and Language Studies’ own Learning and 
Teaching Strategy by: 

• increasing the proportion of student activity on the Virtual Learning Environment (VLE); 

• increasing the proportion of active student learning compared with passive assimilation of 
presentation content (i.e., reading, watching, listening); and 

• integrating digital and information literacy skills into module design. 

Two, three hour LD workshops were developed and delivered using the OULDI LD methodology (Cross, 
Galley, Brasher & Weller, 2012; Galley, 2012). The OULDI methodology makes use of five distinct tools 
to enable module production: the use of Cloudworks to provide a space for users to discuss production 
activity, an activity/pedagogy profiling tool, a module mapping tool, a set of Course Features and 
Information Literacies cards to facilitate and promote discussions about design features and, finally, 
the development of software CompendiumLD which produced visual representations and conceptual 
designs of the emerging module design. These LD workshops were designed to sit alongside regular 
module team meetings where administration, marketing, content and, eventually, the critical appraisal 
of emerging curriculum material were considered. EK313 was first made available to students in 
February 2014. 

Unlike other evaluations of LD (Britain, 2004; Conole et al. 2004), this paper is written from the 
perspective of an academic team member of the module being produced. This is significant in that it is 
the academic team, not the LD team, at the OU which have onward responsibility for the quality 
assurance (QA) and student experience of the module. This paper draws on data from the QA process 
in order to provide a deeper and richer understanding of the impact of LD on the teaching and learning 
experience. 


Analysis 

In order to evaluate the impact of the LD process, three separate data sets were employed. First, open 
ended questionnaires (n=9) were circulated to the academic and academic-related team as well as the 
Faculty’s Associate Dean (Learning & Teaching). Second, to consider the impact of LD on the final 
module “product,” data resulting from the module’s post launch (quality assurance) review, were 
reviewed. This included student (n=6s6) and AL (n=42) survey data (a mixture of closed and open 
questions) to identify how key LD design principles had an impact on the learning and teaching 
experience. Finally, data were compared with the student survey data taken from the module’s 
predecessor, EK310, which had not been through the LD process. 

The data were analysed around four emerging broad “impact” themes which closely aligned with the 
Faculty’s own Learning and Teaching Strategy as outlined in section 3: emphasis and support in the 
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design phase for pedagogy and learning design, increased proportion of student activity on the VLE, 
increased proportion of active student learning, and integration of student digital literacy skills. 


Findings 

Theme 1: Emphasis and Support in the Design Phase for Pedagogy and Learning 
Design 

LD helped to promote a coherent structural curriculum design for EK313 and led to the production of 
both the Module Map (see Figure l) and Activity Planner (see Figure 2) to support and guide production 
activities. These provided a visual representation of the planned learning and teaching experience for 
the new module, setting out the tools and resources for use by students and tutors, and an indication of 
student workload. Second, it was felt that the two design workshops brought about greater collaboration 
among members of the module team compared to module production experiences on other modules 
where LD had not been used. It was felt that the design representations mediated design discussion 
and promoted collaboration and sharing: 

It’s helpful to have people from different backgrounds, different strengths getting together to 
work something through [...] the [learning design] process helped them work together because 
they were all looking at the same things. (Associate Dean, Learning and Teaching) 

There was some indication that frontloading these design workshops at the start of the process only 
might create difficulties in maintaining a strong pedagogical focus on student engagement throughout 
later development, as the module team chair clearly recognised: 

I think there's a danger that you do the thinking really early on and that is productive and then 
that thinking is not threaded through the future decisions and you go through [...] as the pace 
increases, how can we keep that pedagogical focus there? (Academic lead, EK313) 

Data suggested that the student demographic information could have been considered more thoroughly 
during the LD process and that this may have helped the academic team to produce material more 
relevant to all students taking the module: “This course is mandatory for the Youth degree and I felt it 
did not fit in with my aims as per the work based courses” (Youth Justice degree student). 
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'At a glance' Module Map To be' 




Figure l. Module Map for new planned module. 
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Figure 2. Activity Planner for the new planned module. 

Theme 2: Increased Proportion of Student Activity on the VLE 

The initial LD workshops encouraged the module team members to map out curriculum content using 
the Module Map and Activity Planner which show more integrated use of VLE tools such as the block 
review forums and glossary, and other synchronous and asynchronous tools, for example OU Live (our 
on-line tutorial software). The extent to which this level of online study was recognised by students 
varied. For example, there was a clear recognition for the way in which the reader chapters and online 
materials allowed students to consolidate their knowledge and understanding of core themes in EK313: 
“All aspects seemed interconnected so were relevant to gaining more understanding,” and “I found 
working my way through all the online learning activities was interesting and really helped to 
consolidate my learning.” (Student responses). Similar benefits to supporting, and engaging in, online 
learning were recognised by the ALs: “The online material is very well written and clear for students,” 
and “Given that EK313 has mostly online materials, I have certainly found this second presentation 
more user friendly - probably more to do with continuing to share the study materials and discussions 
re access and content with the students”. (AL responses). Despite a recognition for the overall quality 
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of materials and the value of online delivery, the focus on online study was often unexpected for many 
students, as, up to that point, students had completed modules within previous qualification 
frameworks which relied heavily on a traditional printed format: “The fact that everything was on-line 
except one reader signalled a significant change for me, and took some getting used to.” (Student 
response) 

Theme 3: Increased Proportion of Active Student Learning 

Through the design workshops, the module team developed a very clear focus for improving active 
engagement through the development of the Activity Planner that emphasised students’ active 
involvement in online discussion/review forums, analysing information, and dealing with real-life 
examples. However, this focus on active learning was not entirely translated into the module produced. 
For example, the original discussions and mapping of the module at the early LD workshops focussed 
on creativity and experiential learning, whereas the revised planner submitted towards the end of the 
production process suggested there were fewer contact hours and there had been a move back towards 
more assimilative types of activities. 

One way to support productive activities in the Activity Planner was to promote student collaboration 
and communication through tutor group forum activities and “end of block” review fora. In essence, 
this was where students created something concrete online which could then receive feedback from the 
AL in order to feed into the preparation of their next assignment. However, the importance of jointly 
constructing forum threads were not realised by students and ALs. This aspect of the LD did not work 
as originally intended, despite the review fora having a strong pedagogic value for teaching and learning: 

On line forums - lack of use by other students made participating in activities suggested in the 
modules frustrating. In the end I stopped using both the module and tutor group forums which 
was disappointing. (AL) 

Theme 4: Integration of Student Digital Literacy Skills in the Module Design 

During the two initial LD workshops, the module team focussed heavily on designing online activities 
that supported students in acquiring a range of Level 6 academic skills including critical reviews, 
carrying out online literature searches, and synthesising research evidence. The module team members 
perceived that the LD activities enabled the module team to embed digital and information literacy into 
the existing module: “I think things like setting out the Module Map [...] has made us more aware of 
what the existing material was trying to do in terms of the different skills including Information 
Literacy” (Academic module team member). 

The LD activities, however, also revealed the module teams concerns about their own lack of skills or 
experience in designing online activities and embedding these into the curriculum. For students, it was 
not the nature of online study specifically, but the development of study skills that caused some 
tensions. For example, feedback from ALs clearly recognised that acquiring these study skills 
represented a key challenge for many students, particularly those who struggled with basic reading and 
writing skills: 

If students have not the time to study they won’t do it and if they have not the basic skills 
required at level [6] (reading, writing, extracting the main points from a text) they will have 
difficulties with everything. (AL) 
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Discussion 

The findings clearly demonstrate a disconnect between the successful process of incorporating LD into 
the design of the module (as seen in the interview data with module staff) and the actual learning and 
teaching experience of the module (as seen in the data from students and ALs). It appears, therefore, 
that although the module was built on strong pedagogical principles, including shifting from a 
transmission to a participatory model, the final product did not fully reflect the original design plan. 
Some of these innovative design features were not recognised or experienced by the ALs or students. 
This might be because the original design plan was not referred back sufficiently during production so 
the module delivery did not fully reflect the design plan as initially intended. In this sense, it is clear 
that the actual experience of LD on EK313 is somewhat at odds with the perspective that LD should be 
seen as a change process mechanism (Salmon et al., 2008; Anderson et al., 2008) as, for EK313, LD was 
not a continual process but a series of one-off events. 

The LD process did not include a consideration of broader student information (e.g., Socio-economic 
status, educational background, prior study) which would have helped to focus the LD workshops onto 
who were the students who were going to study the module and, therefore, what their needs might be. 
As such, the process did not take into account some of the key social and contextual aspects of the 
student learning experience (Conole, 2013; Laurillard, 2002; Oliver et ah, 2002). Similarly, the LD 
workshop did not consider the module within the context of all the different qualifications it contributed 
to. As a new module within relatively “old” qualification pathways, some of the more innovative aspects 
of teaching promoted by the LD process were at odds with some of the more “traditional” teaching 
methods that went on in the modules that students would have studied prior to EK313. Potentially, 
these early learning design workshops could start at a qualification rather than module level to ensure 
consistency in student learning journey across the entire qualification pathway. 

ALs were involved in the critical reading of materials and a group of students were employed to test two 
units of materials. Crucially this input was accessed after the module had been designed and written, 
so these important stakeholders were not involved in the actual design of the module. The rationale for 
decisions made in relation to pedagogy and design were explored within the LD workshops were not 
sufficiently communicated to ALs and, therefore, an opportunity to benefit from some of the rich and 
professionally relevant feedback from ALs was missed. In this way, the relationship between the 
module’s design, pedagogy, and the student and tutor experience were not fully explored or appreciated. 

The LD process mapped learning design only. It did not sufficiently focus on the FHEQ level of the 
module and how and in what way the pedagogical design could have enabled students working at level 
6. In this way, it could be argued that the LD practice methodology was not systematic enough in terms 
of module development (Beetham & Sharpe, 2007) and the related requirements of supporting the 
needs of students registered on different qualification pathways. 


Conclusions 

This LD pilot has influenced the way that LD continues to be refined, developed, and implemented 
across the whole of the OU. As a result of recommendations made from the EK313 evaluation, the 
University has now moved to a process whereby the OULDI LD is compulsory for all new curriculum 
production. In addition, the new Technology Enhanced Learning (TEL) team work alongside the 
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academic team throughout production, not just at the start, in order to help the team to reflect on the 
nature and level of communicative activities adopted. This insures that the final curriculum product 
can be compared to its original intention, whilst maintaining flexibility to make changes to the product 
where it is felt that there is a strong pedagogical need to do so. In addition, the University is moving 
towards incorporating the student view within LD at a qualification level as part of its on-going student 
consultative work. 
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